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BACKGROUND 

The Challenger Report 



Marshall Space Flight Center 


• Following the Space Shuttle Challenger accident, the 
Rogers Commission reported in 1986: 


- S&MA was not included in technical issue discussions. 


- Inadequate S&MA staffing at MSFC - “Reductions in the safety, 
reliability and quality assurance work force at Marshall and NASA 
Headquarters have seriously limited capability in those vital functions 
(safety program responsibility) to ensure proper communications.” 


“A properly staffed, supported, and robust safety 
organization might well have avoided these faults 
(addressing faults within the S&MA organization 
that contributed to the Challenger Accident).... 99 
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BACKGROUND 

The Columbia Report 



Marshall Space Flight Center 

• Following the Space Shuttle Columbia accident, the 
Columbia Accident Investigation Board (CAIB) reported 
in 2003: 

— “Throughout its history, NASA has consistently 
struggled to achieve viable safety programs and 
adjust them to the constraints and vagaries of 
changing budgets.” 

— “The Board believes that the safety organization, 
due to lack of capability and resources independent 
of the Shuttle Program, was not an effective voice 
in discussing technical issues or mission operations 
pertaining to STS- 107.” 
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BACKGROUND 

The 2006 NASA Exploration Safety Study 


Marshall Space Flight Center 


• The 2006 NASA Exploration Safety Study (NESS) Team 
found that NASA “Safety and Mission Assurance is 
ineffective in carrying out its assigned responsibilities as 
given in the Governance document in many, but not all, 
NASA Centers.” They cited: 

— Lack of leadership 

— Lack of clearly defined lines of authority for action 
— Lack of clearly defined levels of responsibility for S&MA requirements 
— Lack of technical excellence of personnel in the safety disciplines 
— Lack of personnel with domain knowledge 

“All of the above have led to lack of peer level respect from programmatic 
and engineering personnel and has rendered S&MA ineffective . ” 
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BACKGROUND 
The Message from the Past 



Marshall Space Flight Center 


r ^ 

Common 
themes of 
all three 
efforts: 

L J 


• Inadequate resources 

• Lack of discipline expertise 

• Lack of respect by engineering 
peers 

• Lack of inclusion in technical 
decisions 

• Lack of independence 
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Creating the Environment 

The Professional Development Roadmap 



Marshall Space Flight Center 


• Overall Objective - Improve and maintain S&MA expertise 
and skills. 

• Supporting Objectives: 

— Develop a “Professional Development Roadmap” (PDRM) for each of 
the three main S&MA engineering disciplines (Systems Safety, 
Reliability & Maintainability, and Quality Engineering). 

• Provide structured guidance for S&MA engineers to use in their 
efforts to become experts in their field. 

— Identify courses and knowledge that S&MA engineers need in 
order to develop their expertise. 

— Training based on individuals current level of expertise. 

• Provide structured guidance to engineers in the development of their 
annual Individual Development Plan (IDP). 
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Creating the Environment 
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Creating the Environment 

The Professional Development Roadmap 

Marshall Space Flight Center 

• S&MA Discipline Training Roadmaps were expanded 
beyond Systems Safety, Reliability & Maintainability, and 
Quality Engineering to include: 



Industrial 

Safety 

Specialist 
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Creating the Environment 

S&MA Re-organization 

Marshall Space Flight Center 

• Objectives 

— Optimize S&MA organization to best facilitate Shuttle transition in 
2010, successfully support Ares developmental responsibilities, and 
minimize the impacts of the gap between last Shuttle flight and start of 
Ares V Project. 

— Improve leveraging of critical skills and experience between Shuttle and 
Ares. 

— Split technical and supervisory functions to facilitate technical 
penetration. 

— Create Chief Safety and Mission Assurance Officer (CSO) stand-alone 
position for successfully implementation of S&MA Technical Authority. 

— Minimize disruption to customers. 

— Provide early involvement of S&MA leadership team and frequent/open 
communications with S&MA team members and steak-holders. 
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Creating the Environment - S&MA Re-organization 
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Creating the Environment 

S&MA Re-organization 


Marshall Space Flight Center 


QD30 


Vehicle Systems Department 

Department. Manager 
Ares I Level 3 CSO 
Ares V Level 3 CSO 
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Solid Motors Branch 

Branch Chief 

Reusable Solid Rocket Booster CSO 
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QD34 

Launch Sys Integration 
Branch 

Branch Chief 
Shuttle Integration. CSO 
Ares VI CSO 


9/24/2010 


Safety & Mission Assurance 


12 


Creating the Environment 

S&MA Re-organization 

Marshall Space Flight Center 

• Chief Safety and Mission Assurance Officers (CSOs) 

— Are equivalent to Element, Project and Program Chief Engineers. 

— Center Management and Operations (CM&0)Technical Authority (TA) funded. 

- Mainly responsible for project technical down and in. 

- Represent S&MA TA on assigned boards and panels. 

— Responsible for technical quality of organizational products. 

• Department Managers and Branch Chiefs 

— Are the supervisors for the Level III and Level IV CSOs. 

— Can act for their CSOs and implement TA in their CSOs absence. 

- Are CM&O TA funded. 

— Responsible for the care, feeding and staffing of organization. 

— Represent S&MA TA on assigned boards and panels. 

— Responsible for the development of organizational technical products. 
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Creating the Environment 

Post Columbia S&MA Enablers 

Marshall Space Flight Center 

• Agency 

— Created S&MA Technical Authority. 

— Created NASA Safety Center. 

- Created Discipline Fellows ST for S&MA Disciplines (executive technical 
position -in work). 


• MSFC 

— Elevated MSFC S&MA Office to a Directorate. 

— Elevated MSFC S&MA Deputy Director position to Senior Executive 
Service (SES) level. 

— Created senior level engineering SES rotational position (every two years) 
in S&MA - Director for Program Assurance. 

— Elevated Chief Safety and Mission Assurance Officer (CSO). positions to 
grade levels equivalent with MSFC Chief Engineers. 




NASA Headquarters 
Washington, DC 
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Creating the Environment 


Marshall Space Flight Center 
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The S&MA Paradigm Shift 

The System Design Requirements Change 



Marshall Space Flight Center 


• NASA had committed to a major space exploration program, called 
Constellation, which was intended to send crew and cargo to the 
International Space Station (ISS), to the moon, and beyond. 

• In the past, space vehicle designers focused on performance. 

• Lessons learned from the Space Shuttle and other launch vehicles show the 
need to optimize launch vehicles for other system parameters (reliability, 
safety, cost, availability, etc.) besides performance. 

• The Constellation program had, therefore, put in place ambitious 
requirements for reliability, safety, and cost . 

• The new requirements resulted in a paradigm shift on how to design and 
build new launch vehicles which resulted in the creation of an integrated 
Risk-based design environment (e.g. Integrated analyses, disciplines, 
organizations, etc.) and the early involvement of S&MA in the design 
process. 

These lessons will be used to help NASA build future launch vehicles. 
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The S&MA Paradigm Shift 

The System Design Requirements Change 


Marshall Space Flight Center 


Assurance: Making certain that specified activities performed by others are performed in accordance 

with specified requirements. (Upper stage Engine and First Stage). 

Examples of the activities include: 

• Assess Hazard Analyses, FTAs, FMEA/CIL, PRA, etc. 

• Approving Material Review Board dispositions. 

• Performing government inspections, audits, and surveillance. 

• Independent assessments. 

• Evaluating engineering and manufacturing changes, or proposed variances (adaptations, 
deviations, and waivers), for impacts to safety, reliability, and/or quality. 

• Evaluating the disposition of problems, including corrective actions (e.g., PRACA 
problem reports). 

Ill-Line: S&MA activities performed in direct support of the program/project to ensure that the 

program/project will achieve its objectives (Upper Stage and Vehicle Integration). 

Examples of the activities include: 

• Establish and implement S&MA programmatic and technical requirements. 

• Perform Probabilistic Risk Assessments, Reliability Analysis, Integrated System Failure 
Analysis, Hazard Analyses, Fault Tree Analyses, FMEA/CIL, etc. 

• Develop S&MA plans and methodologies. 

• Establish and implement Industrial Safety. 
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Creating the Environment 

The S&MA Project and Engineering Integrated 
Operating Environment Change 




S&MA Integration with Project and 

Engineering 


Crew Safety & 
Reliability 


Crew Safety & 
Reliability Integration 



System Safety FMEA/CIL Ascent Risk Simulation Integrated Aborts 

Working Group Working Group Working Group Assisted Risk Working Group 

(SSWG) (FMEAWG) (ARWG) Analysis (IAWG) 
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Creating the Environment 

The S&MA Project and Engineering Integrated 
Operating Environment Change 

Marshall Space Flight Center 

• S&MA leading the Ares I System Safety Working Group 

— Integrated Failure Modes and Effects Analysis (FMEA) feeds other key analyses 
used to drive the safety and reliability of the Ares I design. 



The Ares I Integrated 
FMEA/CIL serves as input 
data to multiple related 
analyses 


Ascent Risk 
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Creating the Environment 

The S&MA Project and Engineering Integrated 
Operating Environment Change 

Marshall Space Flight Center 

• S&MA leading the Ares I System Safety Working Group 

— Integrated Hazards 

• Identify hazard causes and controls that cross system and element 
boundaries and assure mitigation for the hazard causes 

• Ensure proper communication between Engineering (Design input for 
Hazard Controls) and S&MA - verify safety’s understanding of vehicle 
design and ensure engineering design implementation of potential 
hazards. 

• L2 - address hazards associated with Ares/Orion integrated stack -> 
interface with Level 2 System Engineering and Integration (SE&I). 

• L3 - address hazards associated with Ares vehicle -> Ares VI S&MA 

— Assumed lead role in development of Fault Trees for Controls Hazard 
Report (HR) and Flight Termination System HR to meet Phase 1 
requirements . 



9/24/2010 


Safety & Mission Assurance 


20 


Creating the Environment 

The S&MA Project and Engineering Integrated 
Operating Environment Change 

Marshall Space Flight Center 


S&MA leading the Ares I Ascent Risk Working Group 


i Conceptual Design Phase 

Design & Development Phase 

Operational Phase 

□ Support System 

□ Support Subsystem and 

□ Support System 

Design 

Component Design 

Risk Assessments 

• Integrated system risk 

• Integrated with IPT’s 

• Support launch issues 

modeling and analysis 

• Component reliability 
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• System physics-based 

modeling and analysis 


modeling and analysis 
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• Blast modeling for 

modeling and analysis 


abort risk assessment 




9/24/2010 Safety & Mission Assurance 21 






The Impact - Early Involvement in the Design Process 

Ares I Design Impact (Examples) 


Marshall Space Flight Center 


• Example of S&MA impact on the Ares I Design 

- Influenced the choice of the solution to the Thrust Oscillation issue. Jointly working 
with engineering and Ares I project, S&MA assessed the reliability, quality and safety 
impacts of the various design solutions to the thrust oscillation issue. A lesson learned 
in “integrated failure analysis” from the Shuttle External Tank (ET) foam problem that 
contributed to the Columbia accident (Vehicle Integration). 

- Influenced the design solution to the First Stage-Upper Stage separation issue. Jointly 
working with engineering and Ares I project, S&MA assessed the reliability and safety 
impacts of the various design solutions to the First Stage-Upper Stage separation issue. 
Another lesson learned in “integrated failure analysis” from the Shuttle ET foam 
problem that contributed to the Columbia accident (Vehicle Integration). 

- Recommended pressurization line be moved out of cable tray to reduce risk to Linear 
Shape Charge (LSC) and avionics (Upper Stage). 

- Optimized valve design for reliability and safety for LH2 and L02 pressurization. 

- Identified issue with use of KC fittings in safety-critical applications and approach to 
qualifying fittings as providing two seals (Upper Stage). 

— Influenced the change of LSC initiation timers from percussion to Flexible Confined 
Detonation Cord initiated timers (Flight Termination System). 
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The Impact - Early Involvement in the Design Process 

Ares I Design Impact (Examples) 


Marshall Space Flight Center 


• S&MA In-House Developed Products 

— Vehicle Integration - Crew Safety and Reliability Products 

• Ares I Failure Mode Effects Analysis/Critical Items List (FMA/CIL) 

• Ares I System Safety Analysis Report (Hazard Analysis) 

• Ares I Fault Tree Analysis (FTA) Report 

• Ares I Ascent Risk Analysis (ARA) Report 

• Integrated Aborts Plan 

• Aborts Risk Assessment 

- Upper Stage S&MA Products 

• Safety, Reliability and Quality Plan 

• Failure Mode Effects Analysis 

• System Safety Analysis Report (including Fault Tree) 

• PRA Report 

• Reliability and Maintainability Analysis Report (Reference) 

• Limited Life Items List 
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The Impact - Early Involvement in the Design Process 

Ares I Design Impact (Examples) 


Marshall Space Flight Center 


• Peer Review Products 

— Upper Stage Engine and First Stage Peer Review 

• Quality Assurance Plan 

• System Safety Plan, Safety, Health and Environment Plan 

• Reliability and Maintainability Program Plan 

• Failure Modes and Effects Analysis, Critical Items List, Limited Life Items 

• Reliability Allocations, Predictions and Analysis Report 

• Hazard Fault Tree Analysis Report 


9/24/2010 


Safety & Mission Assurance 


24 



The Impact - Early Involvement in the Design Process 

Ares I Design Reviews (Examples) 


Marshall Space Flight Center 
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Pre-Board Evaluation of PDR Success Criteria 
(71 23.1 A) (4 of 5) 
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7. Any required new technology 
has been developed to an 
adequate state of readiness, or 
backup options exist and are 
supported to make them a viable 
alternative. 


+ Gigabit Ethernet development is see 
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8. The project risks have been 
credibly assessed, and plans, a 
process, and resources exist to 
effectively manage them. 
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Of the 10 Ares PDR 
success criteria, S&MA 
was the only one rated 
entirely “Green” by the 
Pre-Board membership ! 


\ 

I 

A 


K 



sauced 


■W 


9. Safety and mission assurance 
(e.g., safety, reliability, 
maintainability, quality, and EEE 
parts) have been adequately 
addressed in prel i mi nary designs 
and any applicable S&MA 
products (e.g., PRA, system 
safety analysis : and failure 
modes and effects analysis) have 
been approved 


+ Excellent Job of incorporating safety emphasis into the early 
requirements and design phase. 
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S&MA Path to the Future 


Marshall Space Flight Center 



Continue to build accomplishments 
from Ares I. 


Leverage successes from Ares I in 
planning for potential Heavy Lift Launch 
Vehicle conceptual design. 


Leverage the new Agency S&MA 
Technical Excellence Program to develop 
S&MA discipline expertise. 

Program based on MSFC PDRM program. 
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